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Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 

after SIX (6) MONTHS from the mailing date of this communication. . 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )D Responsive to communication(s) filed on . 

2a)Q This action is FINAL. 2b)E3 This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quay/e, 1935 CD. 1 1 , 453 O.G. 213. 

Disposition of Claims 

4) KI Claim(s) 1-24 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) Q Claim(s) is/are allowed. 

6) EI Claim(s) 1-24 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) £3 The specification is objected to by the Examiner. 

10) E3 The drawing(s) filed on 01/22/2001 is/are: a)D accepted or b)E3 objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1 .121 (d). 

1 1) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 
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2.D Certified copies of the priority documents have been received in Application No 



3.D Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
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Art Unit: 2652 

DETAILED ACTION 
Drawings 

1. The drawings are objected to because of spelling error in text of Figure 5, 
element 508: "Compare previous structure to new rule structer to identify first newly 
added branch". Corrected drawing sheets in compliance with 37 CFR 1.121(d) are 
required in reply to the Office action to avoid abandonment of the application. Any 
amended replacement drawing sheet should include all of the figures appearing on the 
immediate prior version of the sheet, even if only one figure is being amended. The 
figure or figure number of an amended drawing should not be labeled as "amended." If 
a drawing figure is to be canceled, the appropriate figure must be removed from the 
replacement sheet, and where necessary, the remaining figures must be renumbered 
and appropriate changes made to the brief description of the several views of the 
drawings for consistency. Additional replacement sheets may be necessary to show the 
renumbering of the remaining figures. The replacement sheet(s) should be labeled 
"Replacement Sheet" in the page header (as per 37 CFR 1 .84(c)) so as not to obstruct 
any portion of the drawing figures. If the changes are not accepted by the examiner, the 
applicant will be notified and informed of any required corrective action in the next Office 
action. The objection to the drawings will not be held in abeyance. 
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Specification 

2. The disclosure is objected to because of the following informalities: spelling error 
in page 13, line 20: "to build binary structures that represents particular rules". 

Appropriate correction is required. 

Claim Objections 

3. Claims 2-5 are objected to because of the following informalities: Spelling error in 
claim 2, line 4: "one recognition value of in the new phrase". Appropriate correction is 
required. The examiner will examine this claim on the merits based on one of the 
prepositions above (of or in). 

Claims 8-1 1 are objected to because of the following informalities: spelling error 
in claim 8, line 2: "where the computer-executable instruction include instructions 
defining". Appropriate correction is required. 

Claim Rejections -35 USC §103 



4. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 
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5. Claims 1-3 and 12-15 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Hutchins (U.S. 4,994,966) in view of Zamora et al. (U.S. 4,887,212). 

Regarding claims 1 and 13, Hutchins discloses a computer-readable medium 
having computer-executable instructions /speech recognition interface for performing 
the steps of: receiving at least one instruction to add a new phrase and semantic 
information associated with the new phrase to a grammar and combining the new 
phrase with at least one other phrase in the grammar to form a single grammar 
structure (Col. 8, Lines 19-47). Hutchins does not disclose associating the semantic 
information with a single recognition value in the new phrase by selecting the first 
possible recognition value in the new phrase that can be associated with the semantic 
information without introducing semantic ambiguity into the grammar structure. 

Zamora et al. (U.S. 4,887,212) disclose associating the semantic information with 
a single recognition value in the new phrase by selecting the first possible recognition 
value in the new phrase that can be associated with the semantic information without 
introducing semantic ambiguity (Col. 8, Lines 20-30 and Lines 44-51, Col. 9, Lines 4-6, 
Col. 10, Lines 39-42, Col. 17, Lines 13-21), to provide faster processing of the phrase 
without sacrificing its semantic information (Col. 2, line 46 - Col. 3, line 2). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to modify the teachings of Hutchins to perform the step of associating 
the semantic information with a single recognition value by selecting the first possible 
recognition value in the new phrase that can be associated with the semantic 
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information without introducing semantic ambiguity as suggested by Zamora et al., the 
motivation being faster processing of the phrase without sacrificing its semantic 
information. 

Regarding claims 2 and 14, Hutchins discloses the step of combining comprises 
adding a branch to the grammar structure to represent at least one recognition value 
of/in the new phrase (Col 8, Lines 19-46). 

Regarding claims 3 and 15, Hutchins discloses associating the semantic 
information with a single recognition value in the new phrase comprises associating the 
semantic information with the first recognition value of the first branch added to the 
grammar structure for the new phrase (Col 9, Line 53 - Col 10, Line 11) [Hutchins 
describes processing the grammatical paths in parallel]. 

Regarding claim 12, Hutchins does not disclose, but Zamora et al. does disclose 
determining that the semantic information cannot be associated with a single 
recognition value without introducing semantic ambiguity into the grammar structure; 
and returning an error message indicating that the semantic information would introduce 
semantic ambiguity into the grammar structure (Col 9, Lines 2-6, Col 10, Lines 64-65, 
Col 26, Lines 39-41). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to modify the teachings of Hutchins to perform the step of determining 
that the semantic information cannot be associated with a single recognition value 
without introducing semantic ambiguity into the grammar structure; and returning an 
error message indicating that the semantic information would introduce semantic 
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ambiguity into the grammar structure as suggested by Zamora et al., the motivation 
being more efficient processing of the phrase to avoid unnecessary processing of 
incorrect grammar structure. 

6. Claims 4-11 and 16-18 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Hutchins (U.S. 4,994,966) in view of Zamora et al. (U.S. 4,887,212) 
as applied to claims 3 and 15 above, and further in view of Horiguchi et al. (U.S. 
6,282,507). 

Regarding claims 4 and 16, the combination of Hutchins and Zamora does not 
explicitly disclose the computer-executable instructions are for performing further steps 
of: identifying existing semantic information that was present in the grammar structure 
before the new phrase was added; and shifting existing semantic information within the 
grammar structure so that the existing semantic information is not on a recognition 
value found in the new phrase. 

Horiguchi et al. teach the computer-executable instructions are for performing 
further steps of: identifying existing semantic information that was present in the 
grammar structure before the new phrase was added and shifting existing semantic 
information within the grammar structure so that the existing semantic information is not 
on a recognition value found in the new phrase (Fig. 26; Col 27, Line 27 - Col 28, Line 
30). Shifting of existing semantic information is pivotal to reduce the semantic 
ambiguity in the grammar structure and it improves the processing time. 



Application/Control Number: 09/767,093 Page 7 

Art Unit: 2652 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to modify the teachings of Hutchins and Zamora by the shifting of the 
semantic information as taught by Horiguchi et al., since this results in avoiding the 
introduction of semantic ambiguity into the grammar structure and reduces the 
processing time. 

Regarding claims 5 and 17, Hutchins discloses (1) shifting existing semantic 
information comprises placing the semantic information on the first recognition value in 
each branch that is parallel to the branch added for the new phrase (claim 5) and (2) the 
semantic information moved by the placement component is found on a transition 
before the transition that differentiates the current sequence of transitions from all other 
transitions (claim 17) (Col 9, Line 53 - Col 10, Line 11). [Hutchins describes processing 
the grammatical paths in parallel]. 

Regarding claims 6 and 7, Hutchins and Zamora do not explicitly disclose the 
computer-executable instructions/speech recognition interface are for performing further 
steps of: providing the grammar structure to a speech recognition engine; receiving an 
indication of a hypothesis word (and confidence score based on probability) recognized 
by the speech recognition engine in part on the grammar structure; and providing the 
semantic information associated with the hypothesis word (and confidence score based 
on probability) to an application. 

Horiguchi et al. teach the computer-executable instructions/speech recognition 
interface are for performing further steps of: providing the grammar structure to a 
speech recognition engine (Fig. 2, element 222); receiving an indication of a hypothesis 



Application/Control Number: 09/767,093 - Page 8 

Art Unit: 2652 

word (and confidence score based on probability) recognized by the speech recognition 
engine based in part on the grammar structure (Fig. 2, element 224); and providing the 
semantic information associated with the hypothesis word (and confidence score based 
on probability) to an application (Fig. 12). 

Therefore it would have been obvious to one ordinarily skilled in the art at the 
time of the invention to modify the teachings of Hutchins and Zamora to provide an 
interface between the grammar structure and the application capable of evaluating the 
grammar structure and furthermore providing this evaluation or confidence score to an 
application as suggested by Horiguchi et al., the motivation being a more confident and 
educated action decision by such receptive application. 

Regarding claims 8 - 1 1 , the combination of Hutchins, Zamora, and Horiguchi et 
al. discloses the claimed computer-executable instruction/speech recognition interface 
includes instructions defining the application and wherein the computer-executable 
instructions/speech recognition interface of the application are for performing a step of 
deciding whether to take an action before complete recognition of a phrase based on 
semantic information provided to the application with a hypothesis word (See Horiguchi, 
Fig. 26). 

Regarding claim 18, the combination of Hutchins, Zamora et al. and Horiguchi et 
al. teaches a speech recognition interface wherein the placement component moves the 
semantic information by associating the semantic information with each transition that 
extends form the same state as the transition that differentiates the current sequence of 
transitions from all other transitions (See Horiguchi, Fig. 26; Col 27, Line 27 - Col 28, 
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Line 30). Shifting of existing semantic information is pivotal to reduce the semantic 
ambiguity in the grammar structure and it improves the processing time. 

7. Claims 19 and 20 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Hutchins (U.S. 4,994,966) in view of Zamora et al. (U.S. 4,887,212) as applied to 
claim 13 above, further in view of Chang et al. (U.S. 6,745,181) and further in view of 
Horiguchi (U.S. 6,282,507). 

Regarding claim 19, the combined teachings of Hutchins and Zamora et al. do 
not disclose (1) a binary grammar-building component that generates a compiled 
representation of the grammar structure and (2) a context-free grammar engine that 
receives the compiled representation of the grammar structure and that provides the 
grammar structure to a speech recognition engine. 

Chang et al. teach a binary grammar-building component that generates a 
compiled representation of the grammar structure (Col 10, Lines 32-46). 

Horiguchi et al. teach a context-free grammar engine that receives a compiled 
representation of a grammar structure and that provides the grammar structure to a 
speech recognition engine (Fig. 25, element 2506 and 2524, Column 27, Lines 8-26, 
Fig. 2). 

Therefore, it would have been obvious to one ordinarily skilled in the art at the 
time of the invention to modify the combination of Hutchins and Zamora et al. with a 
binary grammar-building component that generates a compiled representation of the 
grammar structure as taught by Chang et al. and with a context-free grammar engine as 
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taught by Horiguchi, since (1) the binary format of the grammar is a much more efficient 
means of processing large datasets and ambiguous grammar and (2) to enable the 
processing of the grammar structure. 

Regarding claim 20, the obvious combination above does not include, but 
Horiguchi does disclose the speech recognition interface wherein the context-free 
grammar engine receives a hypothesis transition from the speech recognition engine 
before recognition of a complete utterance, identifies semantic information associated 
with the hypothesis transition, and provides the semantic information to an application 
(Horiguchi et al., Fig. 2, element 222, Fig. 12). 

Therefore, it would have been obvious to one ordinarily skilled in the art at the 
time of the invention to modify the combination above with speech recognition interface y 
wherein the context-free grammar engine receives a hypothesis transition from the 
speech recognition engine before recognition of a complete utterance, identifies 
semantic information associated with the hypothesis transition, and provides the 
semantic information to an application as taught by Horiguchi, to enable faster 
processing of the grammar structure. 

8. Claims 21-24 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Marx et al. (U.S. 6,1 73,266) in view of Schwartz et al. (U.S. 5,621 ,859). 

Regarding claim 21, Marx et al. disclose a computer readable medium having 
computer-executable instructions representing a speech-enabled application capable of 
performing the steps of receiving semantic information associated with a hypothesis 
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word identified by a speech recognition engine before an entire utterance has been 
recognized; and identifying a possible action to take based on the semantic information 
(Abstract, Col 1, Lines 41-54, Col 2, Lines 27-39, Col 11, Lines 32-48), but fails to 
disclose the step of determining whether to take the action before the entire utterance 
has been recognized. 

Schwartz et al. disclose the step of determining whether to take the action before 
the entire utterance has been recognized (Abstract, Col 8, Lines 64-67, Col 8-9, Lines 
25-32). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to modify the teachings of Marx et al. by adding the step of determining 
whether to take the action before the entire utterance has been recognized as taught by 
Schwartz et al., thus reducing the processing time of the application without , adding 
ambiguity in executing the action. 

Regarding claim 22, Marx et al. disclose the further step of receiving a 
confidence score for the hypothesis and wherein determining whether to take the action 
comprises balancing the confidence score with a risk of harm associated with incorrectly 
taking the action (Col 2, Lines 27-39). 

Regarding claim 23, Marx et al. does not disclose, however Schwartz et al. 
discloses determining whether to take the action comprises determining the stability of 
the semantic information of across the sequence of hypotheses (Abstract, Col 11, Lines 
32-54). 
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Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to modify the teachings of Marx et al. by adding determining whether to 
take the action comprises determining the stability of the semantic information of across 
the sequence of hypotheses as taught by Schwartz et al., thus ensuring that the 
application receives the correct information before executing the action. 

Regarding claim 24, Marx et al. disclose determining whether to take the action 
comprises determining not to take the action and instead providing feedback to the user 
identifying the action was not taken (Col 9, Lines 52-65, Col 10, Lines 34-40). 



Conclusion 

9. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Minerva Rivero whose telephone number is (703) 605- 
4377. The examiner can normally be reached on Monday-Friday 8:30 am - 5:00 pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Doris To can be reached on (703) 305-4827. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 
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